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Test  Issues  to  Investigate 


V 


•  Lack  of  consistent  acceptable  FUR  picture  quality 

-  “Appears  as  low  contrast  in  some  situations” 

-  “FUR  looked  pretty  good  when  I  took  off  and  then 
a  while  later  it  didn’t” 


Test  Methodology 


•  Use  a  lab  environment  to  investigate  the  flying  NETD  of  the  most  common 
configuration  of  staring  midwave  FLIR  as  a  function  of  background 


-  Use  available  lab  cameras  as  surrogates  for  flying  system 

-  Use  scene  projector  to  generate  low  backgrounds  with  programmed  NETD  target 

-  Use  FLIR  RTools® software  to  collect  digital 

thermographic  data 

-  Run  scene  projector  with  constant  5°AT  bar  over 
backgrounds  of  20°  to  -40°C 

-  Use  blackbodies  to  collect  data  from  20°C  to  100°C 

-  Analyze  all  data  to  determine  sensor  performance 
dependence  on  quantization,  fixed  pattern  noise,  integration  time  and  background 

-  Extrapolate  to  other  backgrounds  and  sensor  operating  modes  using  curve  fits  and 
established  relationships 


Test  Setup 


Note:  Collimator  has  a  14.4”  focal  length  and  a  4”  aperture,  which  will  project  a 
FOV  of  7.69°  using  the  10242  resistor  array 


5 


Measured  Data  Differs  from  Prediction 


Curve  Fitting  Allows  Noise  Extrapolation 


(a) 


(d) 


Which  Predicts  NETD  at  Higher  Transmissions 

(c) 


Data  Reduction  Takes  Care 


Models  and  Sensors  Need  to  Agree 


-40  -20  0  20  40  60  80  100 

Background  Temp  PC) 
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Quantization  Can  Be  Identified 


smaller  noise  sources  -- 
i.e.,  sensor  is  quantization 
limited  as  indicated  by  flat 
noise  performance 


of  smaller  noise  sources  -- 
in  this  case  the  tell-tale 
sign  of  gain  based  fixed 
pattern  noise 
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Model  Lessons 


I 


•  Improved  model  would  agree  with  measurements 


-  Quantization  noise  is  impactful  at  <14  bits 


-  Sensors  are  not  background  limited 


-  Fixed  pattern  noise  is  far  from  constant 


Test  Equipment  Issues 


•  Appropriate  low  background  test  equipment  does  not  exist 

-  Use  of  scene  projector  is  labor  intensive  and  not  cost  effective 

-  Scene  projector  is  not  really  portable 

-  Scene  projector  was  not  designed  for  radiometric  accuracy  needed  for 
imaging  sensor  work 

-  Scene  projector  lacks  uniformity  and  operability  for  high  resolution  work 

-  Scene  projector  is  thermally  slow  compared  to  blackbody 

-  Scene  projector  has  limited  background  operating  range 
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Observations 


•  Flight  crew  complaints  were  verified 

-  Lack  of  test  equipment  prevented  earlier  discovery 

-  Pervasive  issue 


•  A  scene  projector  is  a  terrible  thing  to  waste 

-  Overly  complex  for  what  should  be  a  simple  task 

-  Lots  of  shortcomings  which  required  workarounds 

•  The  performance  model  is  good  but  the  hardware  doesn’t  agree  with  it 

-  Complex  model 

-  Complex  hardware 

-  Fix  one  or  the  other  or  both 
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Recommended  Path  Forward 


•  Spec  systems  from  -40°C  to  >100°C 

-  Currently  room  temp  only 

•  Fix  sensor  design  to  agree  with  performance  model 

-  Underway  at  Edwards  beginning  in  2008 

-  Prototype  imaging  at  Edwards  since  4  Jan  2012 

-  Future  upgrades  planned 

•  Require  test  data  over  that  range 

-  From  -40°C  to  +100°C 

•  Develop  capable  affordable  test  equipment  to  get  that  data 

-  In  lieu  of  $1 .5M  scene  projector 

-  Using  standard  lab  data  collection  techniques 

-  Underway  at  Edwards  with  2012  SBIR 


Scene  Projector  Replacement 


Where  we  are 


fast,  dynamic  scene 
projection  of  low  quality 
imagery  and  backgrounds 
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Abstract 

Following  an  in-flight  squawk  for  poor  and  unstable  image  quality  during  pre-deployment 
operational  testing  at  Edwards,  the  Benefield  Anechoic  Facility  (BAF)  electro-optics 
laboratory  set  about  to  duplicate  what  was  suspected  as  a  background  temperature 
related  sensitivity  issue  with  the  relevant  family  of  sensors,  specifically  midwave  staring 
FLIRs.  The  specific  system  in  question  was  designed  in  1992  and  fielded  in  1994 
having  relied  on  existing  performance  models  available  at  the  time  which  indicated  the 
performance  should  have  changed  only  slowly  and  tolerably  as  background 
temperature  varied.  However,  experience  at  Edwards  showed  considerable  sensor 
change  in  sensitivity  related  to  background  flux  even  over  the  seemingly  small  20°C 
temperature  variation  at  Edwards  from  late  afternoon  when  the  flights  would  originate 
and  later  in  the  evening  when  data  acquisition  was  undertaken.  Note  that  real 
operational  environments  can  involve  backgrounds  from  -40°C  in  northern  latitudes  to 
well  over  +40°C  in  tropical  and  desert  areas.  These  were  the  limits  used  for  this 
particular  system  and  in  fact  are  still  in  use  today  due  to  design  limitations  which 
prevent  operation  above  +40°C.  A  reasonable  specification  for  sensors  of  this  type  to 
make  them  operationally  suitable  would  be  from  -40°C  to  +100°C  to  accommodate  hot 
environments  populated  with  operating  vehicles.  Note  also  that  such  backgrounds  can 
even  occur  within  a  few  frames  of  video  as  sensors  observe  a  cold  sky  and  then  a  hot 
desert.  Multiple  scenarios  can  stress  sensors  between  these  limits.  Issues  related  to 
duplicating  and  quantifying  sensor  performance  under  these  conditions  were: 

•  No  piece  of  commercial  test  equipment  existed  that  is  specifically  designed  to 
make  sensitivity  measurements  on  tactical  sensors  at  other  than  room 
temperature.  These  measurements  are  known  as  noise  equivalent 
temperature  difference  (NETD)  and  represent  that  temperature  difference  on 
a  large  resolvable  target  at  which  the  sensor  signal  to  noise  ratio  is  1 .  These 
measurements  are  repeatable  and  accurate  at  room  temperature  although 
test  equipment  temperature  difference  measurement  accuracy  in  the  ballpark 
of  5mK  (5  thousandths  of  one  degree  centigrade  or  Kelvin)  or  less  is  needed 
and  is  generally  provided  by  available  equipment  at  room  temperature.  The 


issue  is  making  such  a  measurement  at  low  background  where  frosting  of  the 
target  occurs. 

•  As  a  result  of  test  equipment  limitations  and  rosy  model  predictions,  such 
measurements  have  never  been  made,  with  designers  relying  instead  on  the 
industry  standard  performance  model  to  predict  performance  above  and 
below  room  temperature.  Subsequent  lab  data  on  multiple  sensors  at 
Edwards  showed  this  reliance  to  be  unsubstantiated.  Cited  references 
describe  the  actual  data,  their  implications,  a  pathway  to  solution,  and  finally 
progress  to  date.  That  discussion  is  outside  the  classification  of  this  paper 
and  will  not  be  repeated  here.  This  paper  covers  the  issues  and  needed 
solutions  related  to  getting  accurate  test  data  on  these  sensors.  As  of  this 
writing,  the  Edwards  BAF  EO  lab  is  the  only  facility  in  the  country  to  have 
made  these  measurements. 

To  attempt  to  replicate  flight  conditions,  it  was  suspected  that  the  Edwards  Large 
Format  Resistor  Array  (LFRA)  might  be  able  to  generate  low  temperature  backgrounds 
and  make  sensitivity  measurements  given  its  specified  capability  to  operate  from  room 
temperature  at  +20°C  down  to  low  backgrounds  at  -40°C  since  the  resistor  array  itself  is 
enclosed  in  a  vacuum  dewar,  which  prevents  frosting  when  operated  below  freezing. 
And  the  specific  performance  issue  related  to  the  flight  system  in  question  and  all  others 
in  the  field  today  is  performance  fall-off  below  room  temperature.  This  suspicion  proved 
to  be  correct  with  the  following  caveats: 

•  The  LFRA  uses  an  8  pound  image  plane  heat  sink  for  stability  and  uniformity. 
Such  high  thermal  mass  slows  down  changes  in  background  such  that  a 
move  of  5°C  requires  30  minutes.  Therefore,  making  what  turned  out  to  be  a 
13  point  data  run  from  +20°C  to  -40°C  required  8  hours,  or  about  10  times 
longer  than  such  measurements  would  normally  take  on  a  blackbody. 

•  The  LFRA  has  a  temperature  accuracy  of  ±0.3°C,  which  is  inconsequential  for 
projecting  dynamic  imagery,  but  is  high  when  making  NETD  measurements. 
This  error  was  uncorrectable  and  made  part  of  the  data.  It  was  deemed  to  be 
tolerable  given  the  large  60°C  background  range  we  were  testing  over  and 
further  minimized  by  using  relatively  large  5°C  temperature  spread  over  which 
to  calculate  responsivity. 

•  The  LFRA  has  low  optical  transmission  in  spite  of  its  advertised  90%  per 
window  (2  of  them  or  81%  advertised  overall)  performance.  Lab 
measurements  indicated  a  real  value  of  about  35%,  which  pushed  the  optical 
transmission  of  the  complete  lab  setup  down  to  a  measured  value  of  30.0% 
after  the  91%  collimator  and  94%  sensor  optics  were  included.  Once 
quantified,  this  value  could  be  corrected  in  the  data.  And  it  was  tolerable 
given  the  large  variation  in  sensor  performance  which  was  still  measurable. 
Higher  transmission  would  have  made  performance  variations  more  evident, 
however. 


•  The  LFRA  is  not  radiometrically  calibrated  meaning  that  a  commanded  input 
is  unrelated  to  either  the  absolute  or  relative  temperature  of  the  projected 
imagery.  This  required  considerable  blackbody  correlation  and  projector 
command  adjustment  to  achieve  our  desired  target  differential  temperature  of 
5°C  above  the  background.  Such  a  relatively  high  target  contrast  also  helped 
mask  the  LFRA’s  ±0.3°C  background  accuracy.  Once  established,  however, 
the  target  contrast  remained  constant  over  the  complete  background  range. 

•  The  complexity  and  $1.5M  price  tag  of  LFRA  are  appropriate  for  its  intended 
purpose  of  projecting  dynamic  low  quality  missile  warning  imagery  with 
embedded  hot  spots  up  to  600K,  but  are  inappropriate  for  making  repeated 
mundane  NETD  measurements  with  a  simple  uniform  IK  static  image 
typically  done  with  a  $20k  blackbody  at  room  temperature.  Having  said  that, 
the  LFRA  proved  amazingly  reliable  and  has  never  failed  after  months  of 
repeated  cycling  over  its  entire  temperature  range  and  beyond. 

•  The  LFRA  is  power  hungry  and  requires  5kW  of  3  phase  power  just  for  the 
cooling  system.  Such  high  power  requires  special  wiring  in  the  lab,  and 
detracts  from  its  ability  to  be  moved. 

•  The  LFRA  has  poor  uniformity  and  operability  with  about  3%  of  the  10242 
pixels  being  more  or  less  inoperative.  This  limitation  was  made  tolerable  by 
using  two  large  1320  pixel  areas  in  which  to  average  both  the  target  and 
background  fluxes.  Therefore,  pixel  outages  and  non-uniformities  were 
averaged  out.  This  is  typical  for  NETD  measurements  but  would  disqualify 
LFRA  from  making  high  resolution  measurements. 

•  The  LFRA  has  a  limited  temperature  operating  range  of  +20°C  to  -40°C  owing 
to  the  temperature  sensitivity  of  the  in-dewar  electronics  and  because  of  the 
volatility  of  the  high  performance  coolant.  This  limitation  means  a  separate 
blackbody  test  setup  had  to  be  used  for  temperatures  above  room  temp  up  to 
+100°C.  Some  data  manipulation  was  required  to  generate  continuous  data 
given  the  30.0%  transmission  of  the  scene  projector  compared  to  the  59.9% 
transmission  of  the  blackbody  setup. 

We  were  able  to  work  around  all  the  above  limitations  to  make  use  of  the  only  semi- 
appropriate  piece  of  test  equipment  available  to  us,  the  LFRA.  It  proved  invaluable  in 
obtaining  the  first  data  of  this  kind  ever  taken,  and  it  will  continue  to  support  evolution  of 
improved  sensors  beginning  in  early  2012. 

However,  for  future  sensor  development  and  testing  at  industry  and  government 
facilities,  the  following  test  equipment  requirements  are  evident  from  the  above 
discussion: 


•  The  device  must  be  uniform  and  unpixelated,  i.e.,  a  blackbody. 


•  The  device  must  be  radiometrically  accurate  to  2-3mK  in  both  absolute  and 
differential  terms. 

•  The  power  must  be  standard  lab  at  120V,  single  phase,  20  amp. 

•  The  device  must  be  able  to  make  high  resolution  measurements  over 
background  in  addition  to  NETD. 

•  The  device  must  be  fast  with  temperature  change  capability  on  the  order  of 
2°C  per  minute. 

•  The  device  must  be  simple,  reliable,  and  hence  affordable  with  a  price  tag  no 
greater  than  10%  of  LFRA  with  a  goal  of  $100k. 

•  The  device  must  operate  over  the  entire  specified  background  temperature 
range  for  sensors  of  this  type  or  from  -40°C  up  to  +100°C. 

This  paper  describes  the  actual  testing  methodologies  and  also  describes  progress 
toward  development  of  a  more  appropriate  test  device  and  methodology  to  support 
retrofit  of  existing  sensors  and  development  of  future  ones. 

1.  INTRODUCTION: 

This  is  the  first  paper  describing  the  specific  test  methodologies  and  setups  needed  to 
acquire  this  test  data  in  a  manner  that  would  answer  the  salient  flight  squawk: 

“The  FUR  looked  pretty  good  when  I  took  off  and  a  while  later  it  didn’t.” 

After  observing  recorded  flight  imagery  in  late  2005,  speculation  was  that  the  sensor 
was  experiencing  background  related  sensitivity  issues  to  a  much  greater  extent  than 
the  prevailing  performance  model  would  indicate.  Replicating  that  situation  in  the  lab 
and  quantifying  it  were  the  challenges  at  hand. 

Since  no  blackbody  exists  that  can  operate  for  long  periods  below  freezing,  the  decision 
was  made  to  program  the  scene  projector  to  project  a  representative  NETD  target.  A 
block  diagram  of  the  setup  in  the  lab  is  shown  in  Figure  1 . 

The  uniform  background  target  was  projected  by  the  Santa  Barbara  Infrared  10242 
resistor  array  run  over  its  maximum  recommended  temperature  range  of  +20°C  to  - 
40°C  to  simulate  the  low  background  portion  of  the  anticipated  flight  regime  of  +100°C 
to  -40°C.  The  image  was  projected  through  a  4”  aperture  f/3.6  midwave  refractive 
collimator  to  one  of  the  multiple  InSb  cameras  used  over  the  intervening  5  years  of  data 
gathering  and  design  experimentation.  The  collimator  has  a  14.4”  focal  length  and  a  4” 
aperture,  which  will  project  a  field  of  view  of  7.69°  using  the  10242  resistor  array.  The 
FOVs  of  the  cameras  used  for  this  testing  were  all  less  than  7.32°,  resulting  in  uniform 
background  throughout  the  sensor  FOV.  Thermographic  data  for  bucket  fill,  noise  and 
responsivity  for  the  cameras  were  captured  digitally  using  FUR  Systems’  RTools® 
software  package  or  with  similar  custom  software  as  appropriate. 


Figure  1.  (U)  Scene  projector  test  setup  used  for  low  background  NETD  measurements. 


Figure  2  shows  a  typical  projected  image  as  seen  by  one  of  the  cameras.  This  vertical 
bar  target  was  generated  by  a  special  command  set  programmed  by  Santa  Barbara 
Infrared  to  drive  the  center  100  columns  of  the  10242  resistor  array  to  produce  the 
relatively  large  bar  target  shown.  After  some  experimentation  and  correlation  with  a 
room  temperature  blackbody,  the  temperature  difference  of  the  lower  portion  of  the  bar 
was  determined  to  be  5°C  above  the  background.  Given  the  highly  conductive  nature  of 
heat  exchange  between  the  pixels  and  the  substrate  for  this  projector  (i.e.,  negligible 
radiation  loss),  the  bar  would  stay  at  that  temperature  difference  as  the  substrate 
temperature  was  varied.  This  feature  allowed  a  constant  5°C  target  to  be  used  at  all 
backgrounds  between  +20°C  and  -40°C.  No  other  piece  of  test  equipment  on  the 
market  will  do  this. 
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Figure  2.  Typical  100  column  5°  constant  AT  scene  projector  image  at  20°C  background. 


Background  noise  combined  with  electronic  noise,  fixed  pattern  noise  (FPN)  and 
quantization  noise  is  the  standard  deviation  of  the  bucket  fills  for  the  pixels  inside  the 
left  1320  pixel  region  of  interest  as  calculated  by  RTools®.  Ten  such  readings  were 
averaged  at  each  temperature  to  improve  accuracy.  Responsivity  is  defined,  for  this 
experiment,  as  change  in  bucket  fill  in  digital  counts  per  degree  C.  It  is  determined  by 
the  difference  in  bucket  fill  between  the  two  1320  pixel  sampled  areas,  one  on  the 
background  and  one  on  the  100  column  5°  constant  AT  target  bar,  divided  by  the 
temperature  difference.  The  background  reading  was  also  used  to  define  bucket  fill  at 
the  set  temperature  and  subsequently  to  control  integration  time  to  keep  bucket  fill 
constant  at  80.0%  over  background.  NETD  as  defined  herein  is  the  noise  measurement 
in  units  of  digital  counts  divided  by  the  responsivity  in  units  of  counts  per  degree  C.  The 
noise,  responsivity,  dynamic  range,  and  NETD  performance  curves  of  the  test  FLIRs 
were  plotted  in  absolute  terms  and  also  normalized  to  performance  at  appropriate 
points.  The  absolute  and  normalized  values  were  then  used  to  extrapolate  performance 
up  to  500°C  background  using  curve  fit  coefficients  from  the  measured  data. 

For  blackbody  measurements  above  +20°C,  the  setup  was  similar  to  that  shown  in 
Figure  1  except  the  collimator  and  scene  projector  were  replaced  with  a  blackbody. 
Cameras  used  were  then  focused  on  the  blackbody  at  their  minimum  focus  distance  of 


about  6  feet.  Note  that  the  difference  in  optical  transmission  between  the  30.0%  on  the 
scene  projector  and  the  59.9%  on  the  blackbody  produced  a  requirement  to  correct  the 
blackbody  data  to  an  equivalent  30.0%  transmission  to  generate  a  smooth  performance 
curve  from  -40°C  to  +1 00°C. 

TEST  OBJECTIVE: 

Our  goal  from  the  beginning  of  this  work  in  late  2005  to  first  publication  of  findings  in 
2008  was  to  accurately  relate  our  lab  findings  to  the  pilots’  squawks  so  we  could  explain 
the  relevant  phenomena  and  thereby  plan  effective  redesign  actions.  As  data 
accumulated,  it  became  apparent  that  the  pilots’  squawks  were  real  and  that  six  specific 
design  changes  were  needed  to  correct  them.  Those  changes  and  their  impacts  have 
been  detailed  in  previous  referenced  publications  and  will  not  be  repeated  here.  Suffice 
it  to  say  that  as  of  October  201 1 ,  all  those  changes  have  been  incorporated  into  a 
prototype  sensor,  which  is  operating  as  predicted.  This  paper  details  the  trials  and 
tribulations  involved  in  that  effort.  Those  trials  and  tribulations  are  themselves 
significant  and  will  result  in  new  test  equipment  and  methods. 

TESTS  PERFORMED 

Tests  performed  were: 

1)  Camera  responsivity  and  noise  vs  integration  time  at  backgrounds  of  +100°C  to  -40°C 
using  scene  projector,  blackbody  and  collimator  for  the  prototype  sensor  operating  with 
fully  automatic  integration,  fixed  integration  with  manual  level  correction  simulating  a 
perfect  scene  based  NUC,  and  fixed  integration  without  scene  based  NUC 

•  Develops  table  of  noise  values  at  each  background  and  integration  time  for  each  of 
the  three  operating  modes 

•  Develops  table  of  responsivity  values  at  each  background  and  integration  time  for 
each  of  the  three  operating  modes 

•  Allows  curve  fit  and  noise  parameter  generation  to  extrapolate  data  to  other  optical 
transmissions,  f-numbers,  and  backgrounds 

4.  RESULTS  &  ISSUES: 

Figure  3  shows  a  typical  performance  curve  resulting  from  this  process  along  with  notes 
depicting  modeling  inconsistencies  specifically  related  to  noise  sources.  These 
inconsistencies  required  many  data  corroboration  re-runs,  which  consumed  many  days 
of  lab  time  given  the  slow  thermal  response  of  the  scene  projector  back  plane. 
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Figure  3.  Typical  NETD  performance  measurements  and  extrapolations  over  a  background  range  of  -40°C  to 
+100°C. 

To  relate  our  data  to  operational  systems  of  the  present  and  future,  several  issues  had 
to  be  overcome: 

A.  Low  Optical  Transmission  on  Scene  Projector: 

The  low  29.9%  optical  transmission  of  the  scene  projector  setup  needed  to  be 
extrapolated  to  a  value  representative  of  operational  designs,  which  are  typically  in  the 
70-90  percent  range.  Increasing  optical  transmission  not  only  makes  the  data  more 
relevant,  but  also  resulted  in  non-linearly  increasing  the  effect  of  fixed  pattern  noise 
caused  by  bucket  fill  variation  with  background.  These  bucket  fill  changes  caused  rapid 
increases  in  level  based  fixed  pattern  noise,  which  comprises  80%  of  all  fixed  pattern 
noise.  This  discussion  has  been  covered  in  previous  referenced  publications.  Figure 
4(a)  shows  the  curve  fit  process  that  allowed  total  noise  to  be  quantified  relative  to 
bucket  fill  to  allow  it  to  be  related  to  higher  optical  transmissions  and  hence  lower 
bucket  fills  at  low  background.  The  resulting  noise  equation  provides  good  fit  to  the 
measured  data,  especially  at  lower  background  and  hence  lower  bucket  fill  associated 
with  higher  optical  transmissions.  Figure  4(b)  shows  noise  values  for  two  representative 
optical  transmissions  of  83.1%  and  87%,  indicating  that  the  higher  transmission  allows 
deeper  bucket  fill  depletion  as  background  is  reduced  resulting  in  higher  noise.  Next, 

Figure  4(c)  shows  the  noise  extrapolations  used  for  NETD  predictions,  which  indicate 
that  higher  transmission  with  its  more  rapid  bucket  fill  starvation  at  low  background 
actually  results  in  higher  noise  to  essentially  nullify  the  benefit  of  the  higher 
transmission.  It  should  also  be  noted  that  previous  published  work  determined  that  the 
optimum  optical  transmission  for  this  class  of  sensor  is  approximately  83.1%,  which 


allows  integration  time  to  increase  to  16.6msec  at  -40°C.  Figure  4(d)  shows  the 
extrapolated  measured  performance  compared  to  that  of  a  more  operationally  desirable 
design.  The  details  of  that  design  have  been  published  elsewhere  (ref  5)  and  will  not  be 
repeated  here. 
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Figure  4.  Curve  fitting  process  allows  generation  of  relevant  data  and  comparison  to  alternative  designs. 
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B.  Background  and  Fixed  Pattern  Noise  Behavior: 

The  original  data  could  not  be  correlated  with  the  existing  culturally  accepted 
performance  expectations  for  these  sensors,  namely  that  they  should  exhibit  some 
background  dependence  and  that  fixed  pattern  noise  is  constant  once  corrected  at  room 
temperature.  Figure  5  shows  those  assumptions  to  be  unsupported  by  measured  data. 
As  can  be  seen: 


•  Responsivity  behaves  as  expected  and  declines  consistent  with  dW/dT.  Note 
that  this  effect,  if  normalized  to  room  temperature  as  shown,  is  independent  of 
optical  transmission. 

•  Noise,  however,  is  not  independent  of  optical  transmission  and  behaves  in  a 
completely  different  manner  than  is  conventionally  accepted.  Measured  data 
from  all  sensors  show  this  same  characteristic  where  total  noise,  consisting  of 
quantization,  background,  fixed  pattern,  and  electronic,  departs  from  an  expected 
slope  within  2-3°C  from  the  NUC  point.  Note  that  this  departure  is  dependent  on 
optical  transmission  with  the  data  shown  taken  at  29.9%  on  the  scene  projector. 


Higher  optical  transmission  results  in  more  rapid  bucket  fill  swings  with 
background  and  hence  more  rapid  increases  of  fixed  pattern  noise  away  from  the 
NUC  point,  which  in  this  case  is  at  20°C.  Lab  data  showed  that  the 
straightforward  relationship  between  optics  transmission,  bucket  fill  and 
background  can  be  expressed  as  shown  in  equation  (1). 

(1)  bucket  fill  <<  (bkround  at  temp  T)*(opt.  trans.)+(bkround  at  optics  temp)*(1-opt. 
trans.) 

Once  a  baseline  bucket  fill  is  determined  for  a  given  design  and  background,  this 
relationship  can  then  be  used  to  predict  bucket  fills  for  different  f/#’s,  optics 
temperatures,  or  other  design  parameters. 

This  strong  effect  of  bucket  fill  on  FPN  is  contrary  to  the  existing  standard  performance 
model,  which  assumes  FPN  is  constant  over  background  once  corrected,  which  is 
typically  done  at  room  temperature.  All  lab  data  indicate  this  is  not  the  case  with  FPN 
increasing  sharply  as  the  background  is  moved  away  from  the  NUC  point  by  as  little  as 
2-3°C,  representing  about  a  1%  change  in  bucket  fill  for  83%  transmission  at  room 
temperature  for  example.  Equation  (2)  was  empirically  determined  from  lab  data 
(Figure  4(a))  to  relate  bucket  fill  to  total  noise,  which  for  this  class  of  sensor  is  all  FPN. 

(2)  total  noise  =  0.04*(bucket  fill)2  -  4.3343*(bucket  fill)  +  1 22.9 


Figure  5.  Measured  data  does  not  agree  with  expected  sensor  background  and  fixed  pattern  noise 
performance. 

Many  background  runs  over  the  full  temperature  capability  of  the  scene  projector  were 
required  to  understand  the  relationship  between  the  aforementioned  four  noise  sources. 

The  time  span  required  to  do  this  involved  several  days  of  data  acquisition  spread  over 
several  weeks  with  a  single  run  requiring  about  8  hours  given  the  slow  thermal 
response  of  the  projector  back  plane.  In  the  end  for  this  class  of  sensor,  it  was 
determined  that  fixed  pattern  noise  was  the  dominant  component  with  80%  of  FPN 
being  level  based  and  20%  being  gain  based.  In  addition,  no  evidence  of  background 
noise  influence  could  be  determined.  This  is  contrary  to  the  existing  performance  model 
where  background  noise,  defined  as  the  square  root  of  the  background,  is  assumed  to 
contribute  to  total  noise.  No  such  contribution  was  ever  seen.  See  the  referenced 
publications  at  the  end  of  this  paper  for  detailed  discussion  of  this  issue  and  specific 
solutions  to  it. 

C.  Quantization  Noise  Behavior: 

Over  the  course  of  the  many  weeks  required  to  identify  and  quantify  noise  sources,  it 
finally  became  apparent  that  quantization  noise  could  not  be  ignored.  Figure  6  shows 
representative  noise  performance  for  this  class  of  sensor  with  a  12  bit  compared  to  a  14 
bit  digitizer.  Note  that  in  both  cases,  level  based  fixed  pattern  noise  has  been  removed 
via  a  single  point  level  update  at  each  data  point  to  allow  identification  of  lesser  noises. 

Had  this  not  been  done,  level  based  FPN  would  have  swamped  all  other  sources 
preventing  their  identification.  Clearly,  the  flat  noise  performance  of  the  12  bit  design 


prevents  identification  of  smaller  noise  sources  such  as  gain  based  fixed  pattern, 
electronic,  or  background.  As  can  also  be  seen,  no  indication  of  background 
dependence  is  evident  in  either  case.  A  much  more  complete  discussion  of  these  noise 
components  is  given  in  the  references  and  will  not  be  repeated  here.  For  purposes  of 
this  discussion,  test  equipment  capable  of  operating  below  freezing  would  have  made 
these  measurements  more  evident,  especially  in  the  14  bit  case.  And  as  sensors 
evolve  to  16  bit  performance,  availability  of  such  devices  will  be  critical  so  background 
and  electronic  noise  components  can  be  quantified. 


Figure  6.  Shapes  of  noise  curves  is  clear  indication  of  quantization  noise  (a)  and  of  gain  based  fixed  pattern 
noise  (b). 


D.  Data  Correlation  Between  Test  Setups: 

Given  the  inability  of  the  scene  projector  to  be  operated  above  20°C,  a  separate 
blackbody  setup  is  required  to  collect  data  between  20°C  and  +100°C.  To  do  that,  we 
make  measurements  on  both  test  setups  and  then  correct  the  blackbody  measurements 
for  the  transmission  of  the  projector  or  vice  versa.  Both  setups  use  the  same  sensor 
optics  and  collimator.  To  be  specific,  the  projector  and  blackbody  setups  have  the 
measured  characteristics  shown  in  Table  1. 


Table  1.  Measured  and  inferred  optical  characteristics  of  the  projector  and  blackbody  test  setups. 


Transmission 

Projector  Setup 

-30.0% 

Blackbody  Setup 

-59.9% 

Note  that  the  test  setup  only  allows  a  transmission-emissivity  product  measurement  of 
the  complete  path.  Once  the  transmission  of  the  blackbody  setup  is  established  with  its 
assumed  1.0  emissivity,  then  a  simple  curve  match  to  the  projector  data  can  be  made 
once  the  measured  30%  transmission-emissivity  product  of  the  projector  is  established. 
It  can  be  seen  that  the  transmission  of  the  projector  resistor  array  is  quite  low  at  30%, 
which  will  vary  slightly  on  a  setup  to  setup  basis.  This  value  is  consistent  with  other 
measurements  made  and  published  over  the  last  several  years. 
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Figure  7.  Process  of  adjusting  data  between  two  test  setups  with  different  transmissions  and  emissivities. 

5.  SOLUTIONS: 

Solutions  to  the  issues  presented  in  the  preceding  paragraphs  are  discussed  in  the 
following  section.  Given  the  extensive  data  taken  over  the  last  5+  years,  it  is  clear  that 
both  the  performance  models  and  the  hardware  need  to  improve. 

A.  Model  Issues: 

1.  Model  should  account  for  quantization  noise:  Data  indicate  systems  at  or  below  14 
bits  of  digitization  are  quantization  noise  limited.  This  is  particularly  true  for  12  bit 
designs  running  fixed  integration  time,  which  restricts  bucket  fill  to  no  greater  than 
50%  at  room  temperature  to  prevent  saturation  on  typical  tactical  backgrounds  up  to 
about  +40°C.  This  limitation  prevents  these  systems  from  compensating  for 
electronic  and  background  noises  and  absolutely  precludes  their  achievement  of 
background  limited  noise  performance.  The  performance  models  should  reflect  this. 

2.  Model  should  account  for  non-constant  fixed  pattern  noise:  Data  indicate  non-linear 
influence  of  bucket  fill  changes  on  both  level  based  and  gain  based  fixed  pattern 
noise  with  a  measured  80/20  split,  respectively.  Noise  performance  of  the  various 
sensors  tested  is  consistent  and  shows  a  rapid  noise  increase  for  bucket  fill 
variations  of  about  1%  or  less  up  or  down.  These  data  do  not  support  accepted 
modeling  dictum  which  says  fixed  pattern  noise  is  low  and  constant  once  corrected 
at  room  temperature.  Quite  the  contrary,  both  gain  and  level  based  fixed  pattern 


noises  vary  non-linearly  with  changing  bucket  fill.  The  performance  models  should 
reflect  this. 

3.  Model  should  account  for  non-background  limited  noise  performance:  None  of  the 
sensors  tested  exhibit  influence  of  background  on  their  total  noise  measurements. 
Although  limited  data  is  available  for  constant  bucket  fill  designs,  it  can  be  said  that 
fixed  integration  designs  with  14  bit  digitizers  using  50%  bucket  fill  at  room 
temperature  are  not  background  limited.  The  performance  penalties  associated  with 
this  characteristic  have  been  documented  in  the  referenced  publications.  Additional 
data  from  the  Edwards  constant  bucket  fill  design  will  be  available  in  early  2012. 
That  design  is  expected  to  be  the  first  sensor  to  come  close  to  background  limited 
noise  performance  over  tactically  suitable  backgrounds. 

A.  Test  Equipment  Issues: 

1 .  Use  of  scene  projector  is  labor  intensive  and  not  cost  effective:  The  scene  projector 
was  designed  to  produce  dynamic  images  at  relatively  stable  and  inconsequential 
background  temperature.  It  is,  however,  the  only  piece  of  test  equipment  available 
today  that  can  project  a  static  image  for  different  backgrounds  with  a  slow  rate  of 
change.  Once  changed,  the  background  temperature  of  the  emitter  array  has  been 
demonstrated  to  be  stable  within  ±  0.3°C,  which  is  acceptable  using  the  workaround 
implemented  for  the  testing  described  herein.  The  slow  rate  of  change  of  the  array 
increases  cost  of  testing  by  an  order  of  magnitude  at  least  compared  to  conventional 
blackbody  response  times.  Specifically,  taking  data  from  room  temperature  to  the 
lower  limit  of  the  device  at  -40°C  requires  30  minutes  for  each  of  the  13  data  points 
resulting  in  a  full  8  hour  day  for  two  people.  A  blackbody  type  device  could  perform 
this  run  in  less  than  an  hour  with  better  accuracy. 

2.  Scene  projector  is  not  really  portable:  Given  the  over  half  ton  weight  of  the  device 
and  its  unique  three  phase  and  single  phase  220V  power  requirements,  the  scene 
projector  is  difficult  to  relocate  on  short  notice.  Our  testing  requirements  would  be 
better  served  by  a  device  using  standard  single  phase  120  volt  power  and  weighing 
somewhere  in  the  150  pound  range. 

3.  Scene  projector  was  not  designed  for  radiometric  accuracy:  The  stated  purpose  of 
the  scene  projector  is  to  produce  rapidly  changing  low  resolution,  low  quality 
imagery  at  fixed  slowly  changeable  background  for  consumption  by  non-imaging 
missile  seekers.  As  such,  the  only  radiometric  parameters  required  or  specified 
were  max  emitter  temperature  of  somewhere  in  the  600K+  range  and  an  array 
substrate  range  of  +20°C  to  -40°C.  The  observed  ±300mK  tolerance  on  the 
substrate  temp  is  consistent  with  the  intended  application  of  the  device  for  missile 
seekers  against  inconsequential  backgrounds.  For  our  purposes,  such  a  loose 
tolerance  is  not  inconsequential  but  was  able  to  be  worked  around  by  using  large 
10°C  temperature  spreads  from  which  to  make  calculations.  This  diminished  the 
effect  of  the  variance  although  a  device  with  blackbody  accuracy  in  the  3mK  range 
would  negate  workarounds  and  improve  accuracy. 


4.  Scene  projector  lacks  uniformity  and  operability  for  high  resolution  work:  The  low 
quality  imagery  mentioned  above  refers  to  the  uniformity  and  operability  of  the 
resistor  array,  which  is  about  97%  operable  and  highly  non-uniform.  Note  that 
although  a  non-uniformity  correction  lookup  table  is  part  of  the  device,  the  input 
parameters  cannot  be  better  than  the  imaging  sensor  used  to  measure  them.  The 
end  result  is  that  the  imaging  sensor  under  test  will  be  able  to  see  residual  non¬ 
uniformities  (see  Figure  2).  Worse  yet,  the  non-uniformities  are  not  constant  and 
require  periodic  updates  as  the  array  ages.  This  is  unacceptable  for  high  quality 
image  projection,  but  was  able  to  be  worked  around  by  averaging  large  areas  of  the 
array  and  avoiding  making  high  contrast  or  high  resolution  measurements.  For 
imaging  midwave  sensors,  high  quality  imagery  with  100%  operability  and  uniformity 
below  the  threshold  noise  of  the  sensor  under  test  is  required.  Many  failed 
experiments  have  shown  that  these  uniformity  and  operability  requirements  can  only 
be  achieved  by  a  non-pixelated  blackbody  type  device 

5.  Scene  projector  is  thermally  slow  compared  to  blackbody:  The  scene  projector 
changes  its  substrate  temperature  by  about  5°C  per  30  minutes.  Once  stable  after 
this  time,  data  can  be  acquired.  That  meant  for  a  13  data  point  data  set,  an  entire  8 
hour  shift  for  two  people  was  required.  A  blackbody  will  be  at  least  an  order  of 
magnitude  faster.  Given  the  lack  of  an  alternative  device,  the  slow  response  time  of 
the  scene  projector  was  accommodated. 

6.  Scene  projector  has  limited  background  operating  range:  As  previously  stated,  the 
background  range  of  the  scene  projector  is  limited  to  +20°C  to  -40°C.  This 
restriction  precludes  exploration  of  high  background  conditions  typical  of  desert 
environments  or  high  radiation  environments  such  as  those  populated  with  operating 
vehicles,  fires,  certain  industries,  etc.  This  restriction  required  the  workaround  of 
using  a  blackbody  to  fill  in  the  range  between  +20°C  and  +100°C.  Applying  this 
workaround  required  correction  of  responsivity  for  the  optical  transmissions  of  the 
respective  test  setups  with  the  scene  projector  at  30.0%  and  the  blackbody  at  59.9% 
—  an  inconvenient  and  time  consuming  process.  A  wide  background  range 
blackbody  would  negate  the  need  for  such  data  manipulation  to  get  smooth 
continuous  curves. 

7.  Scene  projector  has  low  optical  transmission:  As  mentioned  above,  the  low  30.0% 
optical  transmission  of  the  scene  projector  strongly  masks  the  effects  of  bucket  fill 
changes  caused  by  background  variation.  These  bucket  fill  changes  cause  a  non¬ 
linear  fixed  pattern  noise  increase  for  bucket  fill  changes  of  less  than  1% 
representing  only  a  2-3°C  change  in  background  from  room  temperature.  The  low 
transmission  masks  this  effect  resulting  in  potentially  lower  data  accuracy  than 
would  a  transmission  of  80%  or  so  representative  of  operational  systems.  This  issue 
could  not  be  worked  around  and  had  to  be  tolerated.  A  simple  blackbody  in  a 
dewar  can  easily  exceed  the  goal  of  80%  transmission. 


6.  PATH  FORWARD: 

The  path  forward  is  clearly  to  build  an  appropriate  piece  of  test  equipment  based  on  the 
requirements  implied  above.  Figure  8  shows  a  comparison  of  where  we  are  today  with 
the  scene  projector  being  the  only  option  for  making  these  kinds  of  measurements 
compared  to  where  we  are  going  in  the  future  with  a  blackbody  type  device  in  a  dewar 
to  allow  fast,  accurate,  uniform,  low  power,  high  transmission,  portable  measurements. 


•  Low  Background  Blackbody 

-  Not  presently  available  due  to: 

-  Lack  of  recognition  of  necessity  of  making  measurements  of  sensitivity  and 
resolution  above  or  below  room  temp  -  especially  below 

-  Incorrect  modeling  which  indicates  only  small  performance  penalty  above 
and  below  room  temp  -  Edwards  work  shows  large  performance  falloff 

-  Only  method  presently  available  is  Santa  Barbara  Infrared  scene  projector 

-  Allows  systemsto  be  spec'd  and  tested  under  realistic  environments 


>$1M,  >1000  lbs,  >5kW 


Figure  8.  Path  forward  to  achievement  of  acceptable  wide  background  test  equipment. 


To  achieve  this  end,  Edwards  launched  a  small  business  innovative  research  (SBIR) 
program  in  201 1  to  begin  the  design  and  development  of  such  a  device.  As  a  result  of 
this  program,  a  prototype  device  should  be  running  in  2013. 
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